- 74 - 

We claim: 

1 . A system for acquiring and reviewing a body of 
information, wherein the body of information includes a 
plurality of segments, each segment representing a defined set 
5 of information in the body of information, the system 
comprising: 

means for acquiring data representing the body of 
information; 

means for storing the acquired datab- 
le first display means for generating a display of a 
first segment of the body of information from data that is 
part of the stored data; 

means for comparing data representing a segment of 
the body of information to data representing a different 
15 segment of the body of information to determine whether, 

according to one or more predetermined criteria, the 
compared segments are related; and 

second display means for generating a display of a 
portion of, or a representation of, a second segment of 
2 0 the body of information from data that is part of the 

stored data, wherein the second display means displays the 
portion or representation of the second segment in 
response to the display by the first display means of a 
first segment to which the second segment is related. 

2 5 2. A system as in Claim 1, wherein the second display 

means displays the portion or representation of the second 
segment substantially coextensive in time with the display of 
the related first segment by the first display means. 

3. A system as in Claim 1, wherein: 

3 0 at least a portion of the body of information is 

represented by audiovisual data; 
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the first segment is represented by audiovisual data; 
the first display means displays an audiovisual 
display of the first segment; and 

the second segment is represented by audiovisual 

5 data. 

4. A system as in Claim 3, further comprising means for 
selecting a segment for which a portion or representation is 
displayed by the second display means, wherein selection of 
such segment causes the first display means to display an 

10 audiovisual display of the selected segment. 

5. A system as in Claim 1, wherein: 
at least a portion of the body of information is 

represented by audiovisual data; 

the first display means displays an audiovisual 
display of the first segment; and 

the second display means displays a text display of 
a portion or representation of the second segment. 

6. A system as in Claim 1, wherein: 
the first display means is an analog display device; 

and 

the second display means is a digital display device. 

7. A system as in Claim 1, wherein: 
the first display means is a television; and 
the second display means is a computer display 

2 5 monitor. 

8. A system as in Claim 1, further comprising means for 
identifying the subject matter content of a segment of the body 
of information, wherein the means for comparing further 
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comprises means for determining the similarity of the subject 
matter content of a segment to the subject matter content of a 
different segment, the predetermined criteria including a 
predefined degree of similarity with respect to which the 
5 relatedness of the compared segments is determined. 

9. A system as in Claim 8, wherein the means for 
determining the similarity of the subject matter of segments 
further comprises means for performing a relevance feedback 
method. 

10 10. A system as in Claim 1, wherein the means for 

acquiring data further comprises means for acquiring television 
broadcast signals . 

11. A system as in Claim 1, wherein the means for 
acquiring data further comprises means for acquiring radio 

15 broadcast signals. 

12. A system as in Claim 1, wherein the means for 
acquiring data further comprises means for acquiring computer- 
readable data files over a computer network from an information 
providing site that is part of that network. 

20 13 , A system as in Claim 1, wherein the means for 

acquiring data further comprises: 

means for acquiring television broadcast signals; and 
means for acquiring computer-readable data files over 
a computer network from an information providing site that 
25 is part of that network. 

14. A system as in Claim 13, wherein: 

the first segment is represented by data produced 
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from the television broadcast signals; and 

the second segment is represented by data from the 
computer- readable data files. 

15. A system as in Claim 1, further comprising means for 
5 identifying an instruction from a user to begin displaying at 

least some of the body of information, wherein the first 
display means begins displaying a segment in response to the 
user instruction. 

16. A system as in Claim 1, wherein the first and second 
10 display means are physically separate. 

17. A system as in Claim 1; wherein the means for storing 
the acquired data, the first display means and the second 
display means are interconnected to a conventional computer bus 
that enables the devices to communicate with each other such 

15 that the devices do not require wire communication over network 
communication lines to communicate with each other. 

18. A system for reviewing a body of audiovisual 
information that can vary with time, the system comprising: 

means for displaying the audiovisual information; and 
20 means for controlling operation of the system, the 

means for controlling being physically separate from the 
means for displaying, the means for controlling including 
a graphical user interface for enabling specification of 
control instructions . 

25 19. A system as in Claim 18, wherein the means for 

controlling is portable. 
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20. A system as in Claim 19, further comprising means for 
2 -way wireless communication between the means for displaying 
and the means for controlling. 

21. A system as in Claim 18, wherein the graphical user 
5 interface includes a playback control region for enabling 

specification of control instructions that control the manner 
in which the audiovisual information is displayed by the means 
for displaying. 



22. A system as in Claim 21, wherein: 



10 



the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
information in the audiovisual informations- 



IB 



the playback control region includes an interface 
that enables selection of one of a plurality of subject 
matter categories; and 



the means for controlling further comprises: 



means for identifying the subject matter 



category of a segment; and 



means for controlling the system to display 



20 



each of the segments that correspond to a selected 



subject matter category. 



23. A system as in Claim 21, wherein: 



25 



the playback control region includes an interface 
that enables variation of the apparent display rate at 
which the audiovisual information is displayed; and 



the means for controlling further comprises means for 
controlling the means for displaying to cause the 
audiovisual information to be displayed at an apparent 
display rate other than a normal display rate. 
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24. A system as in Claim 21, wherein: 

the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
information in the audiovisual information; 

the playback control region includes an interface 
that enables specification of the display of a summary of 
a segment of the audiovisual information; and 

the means for controlling further comprises: 

means for summarizing a segment of the 

audiovisual information; and 

means for controlling the means for displaying 

to cause the summary of the segment to be displayed. 

25. A system as in Claim 21, wherein: 
the playback control region includes: 

an interface that enables specification of a 
pause instruction; and 

an interface that enables specification of a 
resume instruction; and 

the means for controlling further comprises: 

means for identifying a pause instruction from 
a user; 

means for controlling the means for displaying 
to stop the display of the audiovisual information 
in response to identification of the pause 
instruction; 

means for identifying a resume instruction from 
a user; and 

means for controlling the means for displaying 
to restart the display of the audiovisual 
information in response to identification of the 
resume instruction, wherein the audiovisual 
information is displayed at an accelerated rate that 
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is greater than the rate at which the audiovisual 
information was previously displayed, such 
accelerated rate continuing until the display of the 
audiovisual information coincides with the display 
that would have appeared had the display not been 
paused . 

26. A system as in Claim 21, wherein: 

the audiovisual information includes a plurality of 
segments, each segment representing a defined ^ set of 
information in the audiovisual information; 

the playback control region includes an interface 
that enables specification of a termination instruction; 
and 

the means for controlling further comprises: 

means for identifying a termination instruction 

from a user; and 

means for terminating display of the segment 

currently being displayed and beginning display of a 

new segment in response to identification of a 

termination instruction. 

27. A system as in Claim 21, wherein: 

the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
information in the audiovisual information; 

the playback control region includes an interface 
that enables specification of a repeat instruction; and 

the means for controlling further comprises: 

means for identifying a repeat instruction from 

a user; and 

means for repeating the display of the segment 
currently being displayed in response to 
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identification of a repeat instruction. 

28. A system as in Claim 18, wherein the graphical user 
interface includes a map region for providing a description of 
the subject matter content of the audiovisual information and 

5 for enabling specification of control instructions that enable 
navigation within the audiovisual information. 

29. A system as in Claim 28, wherein: 

the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
10 information in the audiovisual information; and 

the map region further identifies a segment of the 
audiovisual information that is currently being displayed. 

30. A system as in Claim 28, wherein: 

the audiovisual information includes a plurality of 
15 segments, each segment representing a defined set of 

information in the audiovisual information; and 

the map region further identifies each segment of the 
audiovisual information that has previously been 
displayed. 

20 31. A system as in Claim 18, wherein: 

the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
information in the audiovisual information; and 

the graphical user interface includes a related 
25 information region for displaying a portion of, or a 

representation of, a segment that is related to a segment 
being displayed by the means for displaying. 

32. A system as in Claim 18, wherein: 
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the audiovisual information includes a plurality of 
segments, each segment representing a defined set of 
information in the audiovisual information; and 

the graphical user interface includes a secondary 
5 information display region for displaying a secondary 

information segment that is related to a segment of the 
audiovisual information that is being displayed by the 
means for displaying. 

33. A system as in Claim 18, wherein the audiovisual 
10 information further comprises the content from one or more news 

programs . 

34. A system for reviewing a body of information, the 
body of information including a first portion that is 
represented by audiovisual data that can vary with time and a 

15 second portion that is represented by text data, comprising: 
a first display device for displaying the first 
portion of information, the first display device 
particularly adapted for generation of a display from 
time-varying audiovisual data; and 

20 a second display device for displaying the second 

portion of information, the second display device 
particularly adapted for generation of a display from text 
data . 

35. A method for acquiring and reviewing a body of 
25 information, wherein the body of information includes a 
plurality of segments, each segment representing a defined set 
of information in the body of information, the method 
comprising the steps of: 

acquiring data representing the body of information; 
3 0 storing the acquired data; 
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generating a display of a first segment of the body 
of information from data that is part of the stored data; 

comparing data representing a segment of the body of 
information to data representing a different segment of 
5 the body of information to determine whether, according to 

one or more predetermined criteria, the compared segments 
are related; and 

generating a display of a portion of, or a 
representation of, a second segment of the body of 
10 information from data that is part of the stored data, 

wherein the display of the portion or representation of 
the second segment is generated in response to the display 
of a first segment to which the second segment is related. 

36. A method for categorizing according to subject matter 
15 an uncategorized segment of a body of information that includes 
a plurality of segments, each segment representing a defined 
set of information in the body of information, one or more 
segments of the body of information having previously been 
categorized by identifying each of the one or more segments 

2 0 with one or more subject matter categories, the method 

comprising the steps of: 

determining the degree of similarity between the 
subject matter content of the uncategorized segment and 
the subject matter content of each of the previously 
25 categorized segments; 

identifying one or more of the previously categorized 
segments as relevant to the uncategorized segment based 
upon the determined degrees of similarity of subject 
matter content between the uncategorized segment and the 

3 0 previously categorized segments; and 

selecting one or more subject matter categories with 
which to identify the uncategorized segment based upon the 
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subject matter categories used to identify the relevant 
previously categorised segments. 

37. A method as in Claim 36, wherein the step of 
determining the degree of similarity is accomplished using a 

5 relevance feedback method. 

38. A method as in Claim 36, wherein the step of 
identifying one or more of the previously categorized segments 
as relevant to the uncategorized segment further comprises the 
steps of : 

10 identifying a plurality of the previously categorized 

segments that are the most similar to the uncategorized 
segment ; 

determining the degree of similarity between each of 
the plurality of previously categorized segments and each 
15 other of the plurality of previously categorized segments; 

for each pair of previously categorized segments of 
the plurality of previously categorized segments having 
greater than a predefined degree of similarity, 
eliminating one of the pair of previously categorized 
20 segments from the plurality of previously categorized 

segments, wherein the previously categorized segment or 
segments remaining after the step of eliminating are 
similar and distinct previously categorized segments; and 
identifying one or more of the similar and distinct 
25 previously categorized segments as relevant previously 

categorized segments. 
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39. A method as in Claim 36, wherein the step of 
selecting one or more subject matter categories further 
comprises selecting the most frequently occurring subject 
matter category or categories associated with the relevant 

5 previously categorized segments. 

40. A system as in Claim 36, wherein the uncategorized 
segment has been acquired from a first data source and the 
previously categorized segment or segments have been acquired 
from a second data source that is different than the first data 

10 source. 

41. A system as in Claim 40, wherein: 

the data acquired from the first data source are 
television or radio broadcast signals; and 

the data acquired from the second data source are 
15 computer- readable data files. 

42. A method for determining whether a first set of 
information represented by a set of data of a first type is 
relevant to a second set of information represented by a set of 
data of a second type, the first and second sets of information 

20 being different from each other, the method comprising the 
steps of: 

deriving a set of data of the second type from the 
set of data of the first type, the derived set of data of 
the second type also being representative of the first set 
25 of information; 

determining the degree of similarity between the set 
of data of the second type representing the second set of 
information and the derived set of data of the second type 
representing the first set of information; and 
3 0 determining whether the first set of information is 
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relevant to the second set of information based upon the 
degree of similarity between the set of data of the second 
type representing the second set of information and the 
derived set of data of the second type representing the 
5 first set of information. 

43. A method as in Claim 42, wherein the first type of 
data is audiovisual data and the second type of data is text 
data . 

44. A method as in Claim 43, wherein the step of 
10 determining the degree of similarity is accomplished using a 

relevance feedback method. 

45. A method as in Claim 42, wherein a plurality of sets 
of information, each different from the other sets of the 
plurality of sets of information, are each represented by an 

15 associated set of data of the second type, the method enabling 
determination of which, if any, of the plurality of sets of 
information represented by a set of data of the second type are 
relevant to the first set of information represented by the set 
of data of the first type, the method further comprising the 

20 steps of: 

determining the degree of similarity between each set 
of data of the second type representing one of the 
plurality of sets of information and the derived set of 
data of the second type representing the first set of 

2 5 information; 

identifying which, if any, of the sets of data of the 
second type representing one of the plurality of sets of 
information have greater than a predefined degree of 
similarity to the derived set of data of the second type 

3 0 representing the first set of information, the sets of 
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data of the second type so identified being termed similar 
sets of data of the second type; 

determining the degree of similarity between each 
similar set of data of the second type and each other 
similar set of data of the second type; 

for each pair of similar sets of data of the second 
type having greater than a predefined degree of 
similarity, eliminating one of the pair of similar sets of 
data of the second type from the set of similar sets of 
data of the second type, wherein the set or sets of 
similar data of the second type remaining after the step 
of eliminating are similar and distinct sets of data of 
the second type; and 

identifying the set or sets of information 
corresponding to one or more of the similar and distinct 
sets of data of the second type as relevant to the second 
set of information . 



46. A method as in Claim 45, wherein the step of 
identifying the relevant set or sets of information further 
20 comprises identifying no more than a predetermined number of 
relevant sets of information, the predetermined number of 
relevant sets of information corresponding to the sets of data 
of the second type having the greatest degree of similarity to 
the derived set of data of the second type. 

2^ 47. A method for identifying the boundaries of segments 

in a body of information, each segment comprising a contiguous 
related set of information in the body of information, wherein 
the body of information is represented by at least a set of 
text data and a set of video data, the method comprising the 

3 0 steps of: 

performing a coarse partitioning method, the coarse 
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partitioning method further comprising the steps of: 

identifying time- stamped markers in the set of 
text data; and 

determining approximate segment boundaries 
5 within the body of information as the times of 

occurrence of the time-stamp markers; 
for each approximate segment boundary, specifying a 
range of time that includes the time of occurrence of the 
approximate segment boundary; 
10 extracting subsets of video data from the ^ set of 

video data that occur during the specified ranges of time; 

performing a fine partitioning method to identify one 
or more breaks in the set of video data; and 

selecting the best break that occurs in each subset 
15 of video data, the time of occurrence of the best break in 

each subset being designated as a boundary of a segment in 
the body of information. 

48, A method as in Claim 47, wherein the step of 
performing a fine partitioning method further comprises 

2 0 identifying the best breaks using a process that includes scene 
break identification. 

49. A method as in Claim 47, wherein the step of fine 
partitioning is performed on the entire set of video data to 
identify all of the breaks in the set of video data. 

25 50. A method as in Claim 47, wherein the step of fine 

partitioning is performed only on the subsets of video data to 
identify only breaks that occur in the subsets. 
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51. A method as in Claim 47, wherein the best break of 
each subset is determined according to the criteria of the fine 
partitioning method used. 

52. A method as in Claim 47, wherein the best break of 
5 each subset is the break occurring closest in time to the time 

of occurrence of the segment boundary in the text data that 
corresponds to that subset . 

53. A method as in Claim 47, wherein the body of 
information is represented by a set of text data, a set of 

10 audio data and a set of video data, the method further 
comprising the steps of: 

ascertaining a synchronization of the audio data and 
the video data; and 

determining the location of the segment boundaries in 
15 the set of audio data using the previously determined 

location of the segment boundaries in the set of video 
data and the synchronization of the audio data and video 
data . 

54 . A method for identifying the boundaries of segments 
20 in a body of information, each segment comprising a contiguous 

related set of information in the body of information, wherein 
the body of information is represented by a set of text data, 
a set of video data, and a set of audio data the method 
comprising the steps of: 
25 performing a coarse partitioning method, the coarse 

partitioning method further comprising the steps of: 

identifying time-stamped markers in the set of 
text data; and 

determining approximate segment boundaries 
3 0 within the body of information as the times of 
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occurrence of the time-stamp markers; 

for each approximate segment boundary, specifying a 
range of time that includes the time of occurrence of the 
approximate segment boundary; 
5 extracting subsets of audio data from the set of 

audio data that occur during the specified ranges of time; 

performing a fine partitioning method to identify one 
or more breaks in the set of audio data; 

selecting the best break that occurs in each subset 
10 of audio data, the time of occurrence of the best break in 

each subset being designated as a boundary of a segment in 
the body of information; 

ascertaining a synchronization of the audio data and 
the video data; and 
15 determining the location of the segment boundaries in 

the set of video data using the previously determined 
location of the segment boundaries in the set of audio 
data and the synchronization of the audio data and video 
data . 

20 55. A method as in Claim 54, wherein the step of 

performing fine partitioning further comprises identifying the 
best breaks using a process that includes pause recognition. 

56. A method as in Claim 54, wherein the step of 
performing fine partitioning further comprises identifying the 

25 best breaks using a process that includes voice recognition. 

57. A method as in Claim 54, wherein the step of 
performing fine partitioning further comprises identifying the 
best breaks using a process that includes word recognition. 

58. A method as in Claim 54, wherein the step of 
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performing fine partitioning further comprises identifying the 
best breaks using a process that includes music recognition. 



59. A computer readable medium encoded with one or more 
computer programs for enabling acquisition and review of a body 
5 of information, wherein the body of information includes a 
plurality of segments, each segment representing a defined set 
of information in the body of information, comprising: 

instructions for acquiring data representing the body 
of information; 
10 instructions for storing the acquired data; 

instructions for generating a display of a first 
segment of the body of information from data that is part 
of the stored data; 

instructions for comparing data representing a 
15 segment of the body of information to data representing a 

different segment of the body of information to determine 
whether, according to one or more predetermined criteria, 
the compared segments are related; and 

instructions for generating a display of a portion 

2 0 of , or a representation of, a second segment of the body 

of information from data that is part of the stored data, 
wherein the display of the portion or representation of 
the second segment is generated in response to the display 
of a first segment to which the second segment is related. 

25 60. A computer readable medium encoded with one or more 

computer programs for enabling categorization according to 
subject matter of an uncategorized segment of a body of 
information that includes a plurality of segments, each segment 
representing a defined set of information in the body of 

3 0 information, one or more segments having previously been 

categorized by identifying each of the one or more segments 
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with one or more subject matter categories, comprising: 

instructions for determining the degree of similarity 
between the subject matter content of the uncategorized 
segment and the subject matter content of each of the 
5 previously categorised segments; 

instructions for identifying one or more of the 
previously categorized segments as relevant to the 
uncategorized segment based upon the determined degrees of 
similarity of subject matter content between the 
10 uncategorized segment and the previously categorized 

segments ; and 

instructions for selecting one or more subject matter 
categories with which to identify the uncategorized 
segment based upon the subject matter categories used to 
15 identify the relevant previously categorized segments. 

61. A computer readable medium encoded with one or more 
computer programs for enabling determination of whether a first 
set of information represented by a set of data of a first type 
is relevant to a second set of information represented by a set 

2 0 of data of a second type, the first and second sets of 

information being different from each other, comprising: 

instructions for deriving a set of data of the second 
type from the set of data of the first type, the derived 
set of data of the second type also being representative 
25 of the first set of information; 

instructions for determining the degree of similarity 
between the set of data of the second type representing 
the second set of information and the derived set of data 
of the second type representing the first set of 

3 0 information; and 

instructions for determining whether the first set of 
information is relevant to the second set of information 
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based upon the degree of similarity between the set of 
data of the second type representing the second set of 
information and the derived set of data of the second type 
representing the first set of information. 

5 62 . A computer readable medium encoded with one or more 

computer programs for enabling identification of the boundaries 
of segments in a body of information, each segment comprising 
a contiguous related set of information in the body of 
information, wherein the body of information is represe.nted by 
10 at least a set of text data and a set of video data, 
comprising : 

instructions for performing a coarse partitioning 
method, the coarse partitioning instructions further 
comprising : 

15 instructions for identifying time-stamped 

markers in the set of text data; and 

instructions for determining approximate 
segment boundaries within the body of information as 
the times of occurrence of the time-stamp markers; 
2 0 instructions for specifying, for each approximate 

segment boundary, a range of time that includes the time 
of occurrence of the approximate segment boundary; 

instructions for extracting subsets of video data 
from the set of video data that occur during the specified 

2 5 ranges of time; 

instructions for performing a fine partitioning 
method to identify one or more breaks in the set of video 
data; and 

instructions for selecting the best break that occurs 

3 0 in each subset of video data, the time of occurrence of 

the best break in each subset being designated as a 
boundary of a segment in the body of information. 
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